Evolving Problems to Learn about Particle Swarm Optimisers and other Search Algorithms

We use evolutionary computation (EC) to automatically find problems which demonstrate the strength and weaknesses of modern search heuristics. In particular we analyse Particle Swarm Optimization (PSO), Differential Evolution (DE) and Covariance Matrix Adaptation–Evolution Strategy (CMA-ES). Each evolutionary algorithms is contrasted with the others and with a robust non-stochastic gradient follower (i.e. a hill climber) based on Newton-Raphson. The evolved benchmark problems yield insights into the operation of PSOs, illustrate benefits and drawbacks of different population sizes, velocity limits and constriction (friction) coefficients. The fitness landscapes made by genetic programming (GP) reveal new swarm phenomena, such as deception, thereby explaining how they work and allowing us to devise better extended particle swarm systems. The method could be applied to any type of optimiser.